{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### AutoGen Advanced\n",
    "\n",
    "This file containes a simple project to learn more on AutoGen AgentChat. The concepts covered in this file consists of:\n",
    "\n",
    "1. Multi-modal conversation\n",
    "2. Structured Outputs\n",
    "3. Using LangChain tools\n",
    "4. Teams\n",
    "5. Model Context Protocol(MCP)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Importing required libraries\n",
    "from io import BytesIO\n",
    "import requests\n",
    "from IPython.display import display, Markdown\n",
    "from pydantic import BaseModel, Field\n",
    "from typing import Literal\n",
    "from PIL import Image\n",
    "from dotenv import load_dotenv\n",
    "\n",
    "# Importing necessary libraries from autogen\n",
    "from autogen_agentchat.messages import TextMessage, MultiModalMessage\n",
    "from autogen_core import Image as AGImage\n",
    "from autogen_ext.models.openai import OpenAIChatCompletionClient\n",
    "from autogen_agentchat.agents import AssistantAgent\n",
    "from autogen_core import CancellationToken\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "load_dotenv(override=True)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### A multi-modal conversation\n",
    "The model has an image as an input and the agent describes the image."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "url = \"https://edwarddonner.com/wp-content/uploads/2024/10/from-software-engineer-to-AI-DS.jpeg\"\n",
    "\n",
    "pil_image = Image.open(BytesIO(requests.get(url).content))\n",
    "img = AGImage(pil_image)\n",
    "img"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "multi_modal_message = MultiModalMessage(content=[\"Describe the content of this image in detail\", img], source=\"User\")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Initializing the gemini client and image describer agent using the client\n",
    "gemini_client = OpenAIChatCompletionClient(model=\"gemini-2.0-flash\")\n",
    "\n",
    "image_describer = AssistantAgent(\n",
    "    name=\"image_describer_agent\",\n",
    "    model_client=gemini_client,\n",
    "    system_message=\"You are good at describing images\"\n",
    ")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### Structured Outputs!\n",
    "\n",
    "Autogen AgentChat makes it easy to use structured outputs."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "class ImageDescription(BaseModel):\n",
    "    scene: str = Field(description=\"Briefly describe the overall scene of the image\")\n",
    "    message: str = Field(description=\"The point that the image is trying to convey\")\n",
    "    style: str = Field(description=\"The artistic stye of image\")\n",
    "    orientation: Literal[\"portrait\", \"landscape\", \"square\"] = Field(description=\"The orientation of the image\")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Image describer agent and calling the agent\n",
    "image_describer = AssistantAgent(\n",
    "    name=\"image_describer_agent\",\n",
    "    model_client=gemini_client,\n",
    "    system_message=\"You are good at describing images in detail\",\n",
    "    output_content_type=ImageDescription\n",
    ")\n",
    "\n",
    "response = await image_describer.on_messages([multi_modal_message], cancellation_token=CancellationToken())\n",
    "reply = response.chat_message.content\n",
    "reply\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "import textwrap\n",
    "print(f\"Scene:\\n{textwrap.fill(reply.scene)}\\n\")\n",
    "print(f\"Message:\\n{textwrap.fill(reply.message)}\\n\")\n",
    "print(f\"Style:\\n{textwrap.fill(reply.style)}\\n\")\n",
    "print(f\"Orientation:\\n{textwrap.fill(reply.orientation)}\\n\")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### Using LangChain tools from AutoGen\n",
    "An advantage of using AutoGen AgentChat is that tools in Langchain, which are updated by the community are available for use by simply calling `LangChainToolAdapter`."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# AutoGen's wrapper:\n",
    "import os\n",
    "os.makedirs(\"output\", exist_ok=True)\n",
    "from unittest import result\n",
    "from autogen_ext.tools.langchain import LangChainToolAdapter\n",
    "\n",
    "# LangChain tools:\n",
    "\n",
    "from langchain_community.utilities import GoogleSerperAPIWrapper\n",
    "from langchain_community.agent_toolkits import FileManagementToolkit\n",
    "from langchain.agents import Tool\n",
    "\n",
    "\n",
    "prompt = \"\"\"Your task is to find a one-way non-stop flight from Nairobi to Las Vegas, through Dubai in December 2025.\n",
    "First search online for promising deals.\n",
    "Next, write all the deals to a file called flights.md with full details.\n",
    "Finally, select the one you think is best and reply with a short summary.\n",
    "Reply with the selected flight only, and only after you have written the details to the file.\"\"\"\n",
    "\n",
    "# Internet Search\n",
    "serper = GoogleSerperAPIWrapper()\n",
    "\n",
    "langchain_serper_tool = Tool(\n",
    "    name=\"internet_search_tool\",\n",
    "    func=serper.run,\n",
    "    description=\"Useful for running internet searches\"\n",
    ")\n",
    "autogen_serper = LangChainToolAdapter(langchain_serper_tool)\n",
    "autogen_tools = [autogen_serper]\n",
    "\n",
    "# Langchain file management tool\n",
    "langchain_file_management_tools = FileManagementToolkit(root_dir=\"output\").get_tools()\n",
    "for tool in langchain_file_management_tools:\n",
    "    autogen_tools.append(LangChainToolAdapter(tool))\n",
    "\n",
    "for tool in autogen_tools:\n",
    "    print(tool.name, tool.description)\n",
    "\n",
    "internet_search_agent = AssistantAgent(\n",
    "    name=\"internet_search_agent\",\n",
    "    model_client=gemini_client,\n",
    "    tools=autogen_tools,\n",
    "    reflect_on_tool_use=True\n",
    ")\n",
    "\n",
    "message = TextMessage(content=prompt, source=\"user\")\n",
    "result = await internet_search_agent.on_messages([message], cancellation_token=CancellationToken())\n",
    "for message in result.inner_messages:\n",
    "    print(message.content)\n",
    "display(Markdown(result.chat_message.content))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Now we need to call the internet_search_agent again to write the file\n",
    "\n",
    "message = TextMessage(content=\"OK proceed\", source=\"user\")\n",
    "\n",
    "result = await internet_search_agent.on_messages([message], cancellation_token=CancellationToken())\n",
    "for message in result.inner_messages:\n",
    "    print(message.content)\n",
    "display(Markdown(result.chat_message.content))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### Team in AutoGen\n",
    "A [team](https://microsoft.github.io/autogen/stable//user-guide/agentchat-user-guide/tutorial/teams.html) is a group of agents that work together to achieve a common goal.\n",
    "\n",
    "AgentChat supports several team presets:\n",
    "\n",
    "- `RoundRobinGroupChat`: A team that runs a group chat with participants taking turns in a round-robin fashion (covered on this page). Tutorial\n",
    "\n",
    "- `SelectorGroupChat`: A team that selects the next speaker using a ChatCompletion model after each message. Tutorial\n",
    "\n",
    "- `MagenticOneGroupChat`: A generalist multi-agent system for solving open-ended web and file-based tasks across a variety of domains. Tutorial\n",
    "\n",
    "- `Swarm`: A team that uses `HandoffMessage` to signal transitions between agents"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "from autogen_agentchat.agents import AssistantAgent\n",
    "from autogen_agentchat.conditions import  TextMentionTermination\n",
    "from autogen_agentchat.teams import RoundRobinGroupChat\n",
    "\n",
    "from autogen_ext.tools.langchain import LangChainToolAdapter\n",
    "from langchain_community.utilities import GoogleSerperAPIWrapper\n",
    "from langchain.agents import Tool\n",
    "\n",
    "serper = GoogleSerperAPIWrapper()\n",
    "internet_search_tool =Tool(\n",
    "    name=\"internet_search_tool\", \n",
    "    func=serper.run, \n",
    "    description=\"Useful for running internet searches\")\n",
    "autogen_serper = LangChainToolAdapter(internet_search_tool)\n",
    "\n",
    "prompt = \"\"\"Find a one-way non-stop flight from Nairobi to Las Vegas in December 2025.\"\"\"\n",
    "\n",
    "# Create primary agent\n",
    "primary_agent = AssistantAgent(\n",
    "    \"primary\",\n",
    "    model_client=gemini_client,\n",
    "    tools=[autogen_serper],\n",
    "    system_message=\"You are a helpful AI research assistant who looks for promising deals on flights. Incorporate any feedback you receive.\",\n",
    ")\n",
    "\n",
    "# Create evaluator agent\n",
    "evaluation_agent = AssistantAgent(\n",
    "    \"evaluator\",\n",
    "    model_client=gemini_client,\n",
    "    system_message=\"Provide constructive feedback. Respond with 'APPROVE' when your feedback is addressed.\",\n",
    ")\n",
    "\n",
    "# Define a termination condition that stops the task\n",
    "text_termination = TextMentionTermination(\"APPROVE\")\n",
    "\n",
    "#Team definition\n",
    "team = RoundRobinGroupChat([primary_agent, evaluation_agent], termination_condition=text_termination, max_turns=10)\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "result = await team.run(task=prompt)\n",
    "for message in result.messages:\n",
    "    print(f\"{message.source}:\\n{message.content}\\n\\n\")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### MCP Introdiction\n",
    "\n",
    "Our first look at the Model Context Protocol from Anthropic -\n",
    "\n",
    "Autogen makes it easy to use MCP tools, just like LangChain tools."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "from autogen_agentchat.agents import AssistantAgent\n",
    "from autogen_ext.models.openai import OpenAIChatCompletionClient\n",
    "from autogen_ext.tools.mcp import StdioServerParams, mcp_server_tools\n",
    "\n",
    "# Get the fetch tool from mcp-server-fetch.\n",
    "fetch_mcp_server = StdioServerParams(command=\"uvx\", args=[\"mcp-server-fetch\"], read_timeout_seconds=30)\n",
    "fetcher = await mcp_server_tools(fetch_mcp_server)\n",
    "\n",
    "# Create an agent that can use the fetch tool.\n",
    "model_client = OpenAIChatCompletionClient(model=\"gpt-4o-mini\")\n",
    "agent = AssistantAgent(name=\"fetcher\", model_client=model_client, tools=fetcher, reflect_on_tool_use=True)  # type: ignore\n",
    "\n",
    "# Let the agent fetch the content of a URL and summarize it.\n",
    "result = await agent.run(task=\"Review edwarddonner.com and summarize what you learn. Reply in Markdown.\")\n",
    "display(Markdown(result.messages[-1].content))"
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": ".venv",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.12.1"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 2
}
